leveraging vision-centric multi-modal expertise
Supplementary Materials: Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection Anonymous Author(s) Affiliation Address email
Moreover, we provide additional visualizations in Section C. Lastly, we explore the potential social The setting of adopted expert-apprentice pairs is depicted in Tab. 1. Let's consider a moving object and analyze the impact of inaccurate motion estimation on its We will provide further details in the subsequent discussion. We hypothesize that this decrease may be attributed to the model's distracted attention BEVDepth: Acquisition of reliable depth for multi-view 3d object detection.
Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection
Current research is primarily dedicated to advancing the accuracy of camera-only 3D object detectors (apprentice) through the knowledge transferred from LiDAR- or multi-modal-based counterparts (expert). However, the presence of the domain gap between LiDAR and camera features, coupled with the inherent incompatibility in temporal fusion, significantly hinders the effectiveness of distillation-based enhancements for apprentices. Motivated by the success of uni-modal distillation, an apprentice-friendly expert model would predominantly rely on camera features, while still achieving comparable performance to multi-modal models. To this end, we introduce VCD, a framework to improve the camera-only apprentice model, including an apprentice-friendly multi-modal expert and temporal-fusion-friendly distillation supervision. The multi-modal expert VCD-E adopts an identical structure as that of the camera-only apprentice in order to alleviate the feature disparity, and leverages LiDAR input as a depth prior to reconstruct the 3D scene, achieving the performance on par with other heterogeneous multi-modal experts. Additionally, a fine-grained trajectory-based distillation module is introduced with the purpose of individually rectifying the motion misalignment for each object in the scene.